Systems and methods for vascular image co-registration

ABSTRACT

A neural network is trained for estimating patient hemodynamic data using a plurality of extravascular imaging data sets and a plurality of intravascular imaging data sets that are each co-registered to a corresponding extravascular imaging data set. A plurality of hemodynamic data sets are provided, each hemodynamic data set co-registered with the corresponding extravascular imaging data set. The neural network learns what hemodynamic data to expect for a given intravascular imaging data set. An intravascular imaging event is subsequently performed in which an intravascular imaging element is translated within a blood vessel of the patient to produce one or more intravascular images. The neural network uses its training to predict hemodynamic values corresponding to the one or more intravascular images from the intravascular imaging event, and the one or more intravascular images are outputted in combination with the predicted hemodynamic values.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority under 35 U.S.C. § 119 ofU.S. Provisional Application No. 63/298,801, filed Jan. 12, 2022, and ofU.S. Provisional Application No. 63/295,722, filed Dec. 31, 2021, theentire disclosures of which are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure pertains to medical imaging, and systems andmethods for medical imaging. More particularly, the present disclosurepertains to systems and methods for vascular imaging includingintravascular imaging and extravascular imaging and co-registration.

BACKGROUND

A wide variety of medical imaging systems and methods have beendeveloped for medical use, for example, use in imaging vascular anatomy.Some of these systems and methods include intravascular imagingmodalities and extravascular imaging modalities for imaging vasculature.These systems and methods include various configurations and may operateor be used according to any one of a variety of methods. Of the knownvascular imaging systems and methods, each has certain advantages anddisadvantages. Accordingly, there is an ongoing need to providealternative systems and methods for vascular imaging and assessment, andco-registration of imaging.

SUMMARY

This disclosure provides alternative medical imaging systems andmethods. An example includes a method for estimating patient hemodynamicdata. The method includes training a neural network, followed bysubsequently obtaining intravascular images for a patient and using thetrained neural network in order to estimate the correspondinghemodynamic data. Training the neural network includes providing aplurality of extravascular imaging data sets to the neural network andproviding a plurality of intravascular imaging data sets to the neuralnetwork, each intravascular imaging data set including intravascularimaging data showing a portion of a blood vessel from a startinglocation to an ending location, each intravascular imaging data setco-registered to a corresponding extravascular imaging data set of theplurality of extravascular imaging data sets. Training the neuralnetwork also includes providing a plurality of hemodynamic data sets tothe neural network, each hemodynamic data set co-registered with thecorresponding extravascular imaging data set of the plurality ofextravascular imaging data sets. The neural network uses the providedplurality of intravascular imaging data sets and the provided pluralityof hemodynamic data sets, each co-registered with the correspondingextravascular imaging data set to learn what hemodynamic data to expectfor a given intravascular imaging data set, thereby creating a trainedneural network. Using the trained neural network with a subsequentpatient includes performing an intravascular imaging event in which anintravascular imaging element is translated within a blood vessel of thepatient from a starting location to an ending location in order toproduce one or more intravascular images. The trained neural networkuses its training to predict hemodynamic values corresponding to the oneor more intravascular images from the intravascular imaging event, andthe one or more intravascular images are outputted in combination withthe predicted hemodynamic values.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets provided while training the neuralnetwork may include intravascular ultrasound data.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets provided while training the neuralnetwork may include optical coherence tomography data.

Alternatively or additionally, at least some of the plurality ofextravascular imaging data sets provided while training the neuralnetwork may include fluoroscopic image data.

Alternatively or additionally, at least some of the plurality ofextravascular imaging data sets provided while training the neuralnetwork may include angiographic image data.

Alternatively or additionally, the angiographic data may includetwo-dimensional angiographic image data.

Alternatively or additionally, the angiographic data may includethree-dimensional angiographic image data.

Alternatively or additionally, the angiographic data may include 3D CTA(three dimensional computed tomography angiography).

Alternatively or additionally, at least some of the plurality ofhemodynamic data sets provided while training the neural network mayinclude pressure data obtained by any hyperemic or non-hyperemic index.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets and at least some of the correspondinghemodynamic data sets may be co-registered using their correspondingpoints in 2D or 3D space on the corresponding extravascular imaging dataset.

Alternatively or additionally, the neural network may include anensemble of neural networks.

Alternatively or additionally, the neural network may include a CNN(convoluted neural network) with transformers.

Alternatively or additionally, the neural network may include amulti-layer neural network.

Alternatively or additionally, the multi-layer neural network mayinclude a hemodynamic term within the loss function.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets provided while training the neuralnetwork may include quantitative data such as lumen borders, vesselborders, side-branch borders, blood speckle density and cardiac cycleparameters, and the quantitative data may be used in training the neuralnetwork.

Alternatively or additionally, the one or more intravascular images fromthe intravascular imaging event include an anatomical landmark, and thepredicted hemodynamic values include a predicted pressure valueproximate the anatomical landmark.

Alternatively or additionally, outputting the one or more intravascularimages in combination with the predicted hemodynamic values may includedisplaying the one or more intravascular images and the predictedhemodynamic values on a graphical user interface of a signal processingunit.

Alternatively or additionally, displaying the one or more intravascularimages and the predicted hemodynamic values on a graphical userinterface of a signal processing unit may include displaying a fullyco-registered display of the predicted hemodynamic values with theintravascular images.

Alternatively or additionally, displaying the one or more intravascularimages and the predicted hemodynamic values on a graphical userinterface of a signal processing unit may include displaying a fullytri-registered display of the predicted hemodynamic values with theintravascular images and a corresponding extravascular image.

Another example includes a method for processing imaging data. Themethod includes providing a plurality of intravascular imaging data setsto a neural network, wherein each intravascular imaging data setincludes intravascular imaging data showing a portion of a blood vessel,co-registered to an extravascular image from a correspondingextravascular imaging data set, from a starting location to an endinglocation. A plurality of hemodynamic data sets are provided to theneural network, wherein each hemodynamic data set includes hemodynamicdata from a corresponding portion of the blood vessel, co-registered toa corresponding extravascular image from the corresponding extravascularimaging data set, from a starting location to an ending location, asrepresented by one of the plurality of intravascular imaging data sets.The neural network uses the provided intravascular imaging data sets andthe corresponding provided hemodynamic data sets, from co-registrationof each data set to the same extravascular image, to learn whathemodynamic data to expect for a given intravascular imaging data set,thereby training the neural network. An intravascular imaging event inwhich an imaging element is translated within a blood vessel from astarting location to an ending location is performed in a new patient inorder to produce one or more intravascular images. The neural networkuses its training to predict hemodynamic values corresponding to the oneor more intravascular images from the intravascular imaging event. Theone or more intravascular images are outputted in combination with thepredicted hemodynamic values.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets may include intravascular ultrasounddata.

Alternatively or additionally, at least some of the plurality ofintravascular imaging data sets may include optical coherence tomographydata.

Alternatively or additionally, at least some of the plurality ofextravascular imaging data sets may include fluoroscopic image data.

Alternatively or additionally, at least some of the plurality ofextravascular imaging data sets may include angiographic image data.

Alternatively or additionally, at least some of the plurality ofhemodynamic data sets may include pressure data obtained by anyhyperemic or non-hyperemic index.

Another example includes a method for processing patient imaging data.The method includes obtaining intravascular imaging data from anintravascular imaging device including an imaging event during atranslation procedure during which the imaging element is translatedwithin a blood vessel from a starting location to an ending location,the intravascular imaging data including one or more intravascularimages. The one or more intravascular images are inputted into a trainedneural network in order to determine a predicted pressure reading foreach of the one or more intravascular images. A series of pressurevalues within the blood vessel corresponding to an intravascularlocation of each of the one or more extravascular images are calculated,and a pressure ratio is calculated based on the series of pressurevalues.

Alternatively or additionally, the method may further include outputtingthe intravascular imaging data and the calculated pressure correspondingto a point within the blood vessel.

Alternatively or additionally, the method may further include obtainingextravascular imaging data including one or more extravascular images,and co-registering the intravascular imaging data with the extravascularimaging data in order to determine an intravascular location of each ofthe one or more extravascular images.

Alternatively or additionally, the method may further include outputtingthe co-registered extravascular imaging data in combination with theintravascular imaging data and the calculated pressure pointcorresponding to a point within the blood vessel.

Alternatively or additionally, obtaining extravascular imaging data mayinclude obtaining extravascular imaging data corresponding to the bloodvessel from the starting location to the ending location.

Alternatively or additionally, the intravascular imaging data mayinclude intravascular ultrasound data.

Alternatively or additionally, the intravascular imaging data mayinclude optical coherence tomography data.

Alternatively or additionally, the extravascular imaging data mayinclude fluoroscopic image data.

Alternatively or additionally, the extravascular imaging data sets mayinclude angiographic image data.

Another example includes a method for processing imaging data. Themethod includes encoding physical features from a plurality of IVUSframes produced during an IVUS pullback run. PRI (physiology restingindex) pullback data is collected. Angiography imaging data is collectedand is co-registered with the PRI pullback data. The IVUS frames areco-registered with the angiography imaging data in order to co-registerthe PRI pullback data with the IVUS frames. The co-registered IVUSframe, angiography imaging data and PM pullback data are used to train aneural network how to predict PRI data based on a subsequent IVUSpullback run. Subsequently, a new IVUS pullback run is executed in orderto provide new IVUS pullback run data that includes a plurality of IVUsframe to the neural network so that the neural network can computepredicted PRI values for each IVUS frame.

Alternatively or additionally, the method may further includeco-registering the new IVUS pullback run data with a correspondingangiography run.

The above summary of some embodiments is not intended to describe eachdisclosed embodiment or every implementation of the present disclosure.The Figures, and Detailed Description, which follow, more particularlyexemplify these embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may be more completely understood in consideration of thefollowing detailed description in connection with the accompanyingdrawings, in which:

FIG. 1 is a schematic overview of training and using a neural networkfor predicting hemodynamic values based on intravascular imaging data;

FIG. 2A is a flow diagram showing an illustrative method of estimatingpatient hemodynamic data;

FIG. 2B is a flow diagram showing an illustrative method of training aneural network as part of the method of FIG. 2A;

FIG. 2C is a flow diagram showing an illustrative method of using atrained neural network as part of the method of FIG. 2A;

FIG. 3 is a flow diagram showing an illustrative method of processingimaging data;

FIG. 4 is a flow diagram showing an illustrative method of processingpatient imaging data;

FIG. 5 is a flow diagram showing an illustrative method of processingimaging data;

FIG. 6 is a schematic view of a illustrative model;

FIG. 7 is a schematic view of a illustrative model;

FIG. 8 is a schematic view of an illustrative model;

FIG. 9 is a schematic illustration of an exemplary system for use invascular imaging co-registration;

FIG. 10 is a schematic illustration of an exemplary intravascularimaging catheter, shown in partial cross-sectional view;

FIG. 11 is a schematic illustration of the distal portion of theexemplary intravascular imaging catheter of FIG. 10 , shown incross-section.

While the disclosure is amenable to various modifications andalternative forms, specifics thereof have been shown by way of examplein the drawings and will be described in detail. It should beunderstood, however, that the intention is not to limit the invention tothe particular embodiments described. On the contrary, the intention isto cover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the disclosure.

DETAILED DESCRIPTION

For the following defined terms, these definitions shall be applied,unless a different definition is given in the claims or elsewhere inthis specification.

All numeric values are herein assumed to be modified by the term“about”, whether or not explicitly indicated. The term “about” generallyrefers to a range of numbers that one of skill in the art would considerequivalent to the recited value (e.g., having the same function orresult). In many instances, the terms “about” may include numbers thatare rounded to the nearest significant figure.

The recitation of numerical ranges by endpoints includes all numberswithin that range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and5).

As used in this specification and the appended claims, the singularforms “a”, “an”, and “the” include plural referents unless the contentclearly dictates otherwise. As used in this specification and theappended claims, the term “or” is generally employed in its senseincluding “or” unless the content clearly dictates otherwise.

It is noted that references in the specification to “an embodiment”,“some embodiments”, “other embodiments”, etc., indicate that theembodiment described may include one or more particular features,structures, or characteristics. However, such recitations do notnecessarily mean that all embodiments include the particular features,structures, or characteristics. Additionally, when particular features,structures, or characteristics are described in connection with oneembodiment, it should be understood that such features, structures, orcharacteristics may also be used connection with other embodimentswhether or not explicitly described unless clearly stated to thecontrary.

The following detailed description should be read with reference to thedrawings in which similar elements in different drawings are numberedthe same. The drawings, which are not necessarily to scale, depictillustrative embodiments and are not intended to limit the scope of theinvention.

A number of different medical imaging modalities may be used to evaluateor treat blood vessels. Two general types of imaging modalities includeextravascular imaging modalities and intravascular imaging modalities.This disclosure relates to the use and co-registration of thesemodalities.

Extravascular imaging modalities, such as various forms of radiologicalimaging, provide extravascular imaging data of a portion of a bloodvessel. Some examples include angiography or fluoroscopy imagingmodalities, such as two-dimensional angiography/fluoroscopy;three-dimensional angiography/fluoroscopy; or computer tomographyangiography/fluoroscopy. Angiography typically involves rendering aradiological view of one or more blood vessels, often with the use ofradiopaque contrast media. An angiographic image can also be viewed realtime by fluoroscopy. In general, fluoroscopy uses less radiation thanangiography, and is often used to guide medical devices includingradiopaque markers within or through vessels. Extravascular imaging dataof blood vessels may provide useful information about the blood vessel,the anatomy or the location or positioning of devices within the bloodvessel or anatomy. For example, extravascular imaging data (e.g.angiograms) may provide a comprehensive overall image or series ofimages or a video of the blood vessel(s) of interest, and may provide a“roadmap” with a good temporal resolution for the general assessment ofthe blood vessel(s) or navigation of devices within blood vessels.

Intravascular imaging modalities provide intravascular imaging data of aportion of a blood vessel. Some examples of intravascular imagingmodalities include intravascular ultrasound (IVUS) and optical coherencetomography (OCT). These modalities typically include imaging the vesselitself using a device-mounted intravascular probe including an imagingelement disposed within the vessel. Several types of device systems havebeen designed to track through a vasculature to provide intravascularimage data. These can include, but are not limited to, intravascularultrasound (IVUS) devices and optical coherence tomography (OCT) devices(e.g. catheters, guidewires, etc). In operation, intravasculardevice-mounted probes including an imaging element are moved along ablood vessel in the region where imaging is desired. As the probe passesthrough an area of interest, sets of intravascular image data areobtained that correspond to a series of “slices” or cross-sections ofthe vessel, the lumen, and surrounding tissue. These devices may includeradiopaque material or markers. Such markers are generally positionednear a distal tip or near or on the probe. Therefore, the approximatelocation of the imaging probe or imaging element can be discerned byobserving the procedure on either a fluoroscope or an angiographic imageor images. Typically, such imaging devices are connected to a dedicatedprocessing unit or control module, including processing hardware andsoftware, and a display. The raw image data is received by the console,processed to render an image including features of concern, and renderedon the display device. Intravascular imaging data of blood vessels mayprovide useful information about the blood vessel that is different fromor in addition to the information provided by the extravascular imagingdata. For example, intravascular imaging data may provide data regardingthe cross-section of the lumen, the thickness of deposits on a vesselwall, the diameter of the non-diseased portion of a vessel, the lengthof diseased sections, the makeup of deposits or plaque on the wall ofthe vessel, assessment of plaque burden or assessment of stentdeployment.

These two general types of imaging modalities provide different imagingdata, and therefore may be complimentary to each other. As such, incertain circumstances, it may be desirable to provide or use bothgeneral types of medical imaging modalities to evaluate or treat bloodvessels. Additionally, it may be useful for the locations of theacquired intravascular imaging data/images to be correlated with theirlocations on the vessel roadmap obtained by the extravascular imagingdata/images. It may be useful to coordinate or “register” (e.g.co-register) the imaging data rendered by the two different modalities.It may also be useful to display the co-registered extravascular imagingdata and intravascular imaging data together, for example, on a commondisplay monitor. Some example embodiments disclosed herein may includeor relate to some or all of these aspects.

In accordance with some embodiments of the present disclosure, examplemethod(s), system(s), device(s), or software are described herein. Theseexamples include image data acquisition equipment and data/imageprocessors, and associated software, for obtaining and registering (e.g.co-registering) imaging data rendered by the two distinct imagingmodalities (e.g. extravascular imaging data and intravascular imagingdata). Additionally, or alternatively, example method(s), system(s) orsoftware may generate views on a single display that simultaneouslyprovides extravascular images with positional information andintravascular images associated with an imaging probe (e.g., an IVUS orOCT probe) mounted upon an intravascular device.

Hemodynamic data can be useful in ascertaining the health of a patient.In some instances, hemodynamic information such as but not limited topressure data can be helpful in ascertaining the health of the patient'svascular system. A variety of systems for obtaining hemodynamic data maybe used. These systems for obtaining hemodynamic data can require one ormore pullback runs in order to obtain data. In some instances, it may beuseful to provide hemodynamic data without requiring any additionalpullback runs, or any other processes or techniques for obtainingpressure information and/or other hemodynamic data.

FIG. 1 provides a schematic overview of an illustrative system 10 bywhich a neural network 12 may be trained in order to provide estimatedhemodynamic values corresponding to particular intravascular images. Theneural network 12 may be any of a variety of different types of neuralnetworks. In some cases, the neural network 12 may represent a singleneural network or a plurality of neural networks. In some instances, theneural network 12 may be manifested within a cloud-based server, forexample. The neural network 12 may represent a CNN (convoluted neuralnetwork) that includes one or more transformers. In some cases, theneural network 12 may include a multi-layer neural network. These arejust examples.

The neural network 12 may be adapted to learn. In some instances, theneural network 12 may be considered as including AI (artificialintelligence) and may optionally be considered as being capable of ML(machine learning). In order to train the neural network 12, the neuralnetwork 12 may be provided with preexisting data with which the neuralnetwork 12 can learn. In some instances, the neural network 12 may betrained how to associate particular hemodynamic properties or valueswith corresponding intravascular images. The neural network 12 may beprovided with a plurality of intravascular image data sets that, as willbe discussed with respect to FIGS. 9 to 11 , be provided from a varietyof different imaging modalities such as but not limited to intravascularultrasound and optical coherence tomography data. The neural network 12may be provided with a plurality of extravascular image data sets 16.The extravascular image data sets 16 may include fluoroscopic image dataand/or angiographic image data. Examples of angiographic image datainclude but are not limited to 2D (two-dimensional) angiographic data,3D (three-dimensional) angiographic data and 3D CTA (three dimensionalcomputed tomography angiography).

Each of the plurality of extravascular image data sets 16 that areprovided to the neural network 12 may be co-registered with acorresponding one of the plurality of intravascular image data sets 14,such as if a particular intravascular image data set 14 corresponds to aparticular intravascular image data acquisition session (such as animaging pullback run) for a particular portion of a blood vessel, from aparticular starting point to an particular ending point, for a patient,and the corresponding extravascular image data set 16 corresponds toextravascular image data of the same portion of the same patient's sameblood vessel, from the same starting point to the same ending point. Insome cases, the portion of the patient's anatomy represented by aparticular intravascular image data set 14 and that represented by aparticular extravascular image data set 16 may not coincide exactly, butmay overlap. In either event, each intravascular image data set 14 maybe co-registered with the corresponding extravascular image data set 16,as indicated at block 18. Methods of co-registering the intravascularimage data sets 14 and the extravascular image data sets 16 will bedetailed with respect to FIGS. 9 to 11 , to be discussed subsequently.

The neural network 12 may be provided with preexisting hemographic datasets 20. In some cases, a particular hemographic data set will representhemographic data, such as but not limited to pressure data, for aparticular patient. In some cases, each hemographic data set 20 willcorrespond to one or more pressure measurements taken at particularlocations within a particular patient's particular blood vessel. In someinstances, the one or more pressure measurements will correspond toparticular locations within the particular blood vessel that coincidewith the anatomy represented by a particular extravascular image dataset 16. In other words, each of the hemodynamic data sets 20 may beco-registered with a corresponding extravascular image data set 16, asindicated at block 22.

It will be appreciated that by co-registering each of the intravascularimage date sets 14 with a corresponding one of the extravascular imagedata sets 16, and that by co-registering each of the hemodynamic datasets 20 with a corresponding one of the extravascular image data sets16, the neural network 12 is able to ascertain correlations betweenintravascular image data, extravascular image data and hemographic data.As a result, the neural network 12 is able to learn, by processing anumber of intravascular image data sets 14, a number of correspondingextravascular image sets 16 and a number of corresponding hemographicdata sets 20, and by being given or otherwise determining aco-registration between the intravascular data and the extravasculardata, a co-registration between the extravascular data and thehemographic data, and thus a co-registration between the intravasculardata and the hemographic data, how to estimate or predict hemographicdata such as pressure measurements as a result of what the neuralnetwork 12 is seeing in a particular intravascular image or images.

At least some of the intravascular image data sets 14, at least some ofthe extravascular image data sets 16 and at least some of thehemodynamic data sets 20 may represent historical data that has beenpreviously obtained and saved. At least some of the intravascular imagedata sets 14, at least some of the extravascular image data sets 16 andat least some of the hemodynamic data sets 20 may represent datacaptured from volunteers who undergo these imaging processes in order tocontribute useful data for training the neural network 12. At least someof the intravascular image data sets 14, at least some of theextravascular image data sets 16 and at least some of the hemodynamicdata sets 20 may represent patient data that can be independentlycaptured for research purposes as patients undergo intravascularimaging, extravascular imaging and hemodynamic measurements for any of avariety of different clinical purposes.

As a result of training, the neural network 12 may be considered ashaving evolved into a trained neural network 24. In this, thedistinction between the neural network 12 and the trained neural network24 may not simply be binary, i.e., the neural network 12 turns into thetrained neural network 24 upon completion of sufficient training. Insome instances, training may continue indefinitely. A neural networkthat is considered to have been trained may be periodically tested, suchas by providing the neural network 24 with intravascular data whilehemodynamic data obtained from the same patient, from the same anatomyand at essentially the same time, may be used as a check against theestimated hemodynamic measurements provided by the trained neuralnetwork 24. If the actual hemodynamic measurements are close to thepredicted hemodynamic measurements, this can be construed as anindication that the trained neural network 24 is indeed well trained.If, however, there are discrepancies or even substantial discrepanciesbetween the actual hemodynamic measurements and to the predictedhemodynamic measurements, this can be construed as an indication thatthe trained neural network 24 may benefit from additional training.

Once the neural network 12 has been trained into the trained neuralnetwork 24, the trained neural network 24 may be used to provideestimated hemodynamic values in response to an intravascular pullbackrun such as but not limited to an IVUS (intravascular ultrasound)pullback run. Performing an intravascular pullback run can provide asource of intravascular images 26. Feeding the intravascular images 26to the trained neural network 24 can result in predicted hemodynamicvalues 28. The trained neural network 24 will have learned, throughtraining, what hemodynamic values 28 have historically resulted from aparticular set of parameters defining an intravascular image. Forexample, a particular type and size of obstruction within a blood vesselhistorically results in particular changes in pressure readings. Oncethe trained neural network 24 determines the estimated hemodynamicvalues, the intravascular images and the corresponding predictedhemodynamic values may be outputted onto any available screen, asindicated at block 30. In some cases, for example, the intravascularimages and the corresponding predicted hemodynamic values may beoutputted via a computer, such as but not limited to the computersystem/sub-system 130 described with respect to FIG. 9 .

FIG. 2A, 2B and 2C are flow diagrams that in combination provide anillustrative method 32 for estimating patient hemodynamic data. FIGS. 2Band 2C provide the detail not outlined in FIG. 2A. The method 32includes training a neural network (such as the neural network 12), asindicated at block 34, and using the trained neural network (such as thetrained neural network 24) with a subsequent patient, as indicated atblock 26.

In some instances, the neural network may include an ensemble of neuralnetworks. The neural network may include a CNN (convoluted neuralnetwork) with transformers. In some cases, the neural network mayinclude a multi-layer neural network. The multi-layer neural networkmay, for example, include a hemodynamic term within the loss function.

FIG. 2B shows details regarding the method 34 for training the neuralnetwork. A plurality of extravascular imaging data sets are provided tothe neural network, as indicated at block 34 a. A plurality ofintravascular imaging data sets are provided to the neural network, eachintravascular imaging data set including intravascular imaging datashowing a portion of a blood vessel from a starting location to anending location, each intravascular imaging data set co-registered to acorresponding extravascular imaging data set of the plurality ofextravascular imaging data sets, as indicated at block 34 b. At leastsome of the plurality of intravascular imaging data sets provided whiletraining the neural network include intravascular ultrasound data. Atleast some of the plurality of intravascular imaging data sets providedwhile training the neural network include optical coherence tomographydata.

A plurality of hemodynamic data sets are provided to the neural network,each hemodynamic data set co-registered with the correspondingextravascular imaging data set of the plurality of extravascular imagingdata sets, as indicated at block 34 c. The neural network uses theprovided plurality of intravascular imaging data sets and the providedplurality of hemodynamic data sets, each co-registered with thecorresponding extravascular imaging data set to learn what hemodynamicdata to expect for a given intravascular imaging data set, therebycreating a trained neural network, as indicated at block 34 d. In someinstances, at least some of the plurality of hemodynamic data setsprovided while training the neural network comprise pressure dataobtained by any hyperemic or non-hyperemic index.

In some cases, at least some of the plurality of extravascular imagingdata sets provided while training the neural network includefluoroscopic image data. At least some of the plurality of extravascularimaging data sets provided while training the neural network may includeangiographic image data. The angiographic data may includetwo-dimensional angiographic image data, for example, and/or may includethree-dimensional angiographic image data. In some instances, at leastsome of the angiographic data may include 3D CTA (three dimensionalcomputed tomography angiography).

In some instances, at least some of the plurality of intravascularimaging data sets and at least some of the corresponding hemodynamicdata sets may be co-registered using their corresponding points in 2D or3D space on the corresponding extravascular imaging data set. In someinstances, at least some of the plurality of intravascular imaging datasets provided while training the neural network include quantitativedata such as lumen borders, vessel borders, side-branch borders, bloodspeckle density and cardiac cycle parameters, and the quantitative datais used in training the neural network.

FIG. 2C shows details regarding the method 36 for using the trained theneural network (such as the trained neural network 24) with a subsequentpatient. An intravascular imaging event is performed in which anintravascular imaging element is translated within a blood vessel of thepatient from a starting location to an ending location in order toproduce one or more intravascular images, as indicated at block 36 a.The trained neural network uses its training to predict hemodynamicvalues corresponding to the one or more intravascular images from theintravascular imaging event, as indicated at block 36 b. The one or moreintravascular images are outputted in combination with the predictedhemodynamic values, as indicated at block 36 c.

In some instances, the one or more intravascular images from theintravascular imaging event include an anatomical landmark and thepredicted hemodynamic values include a predicted pressure valueproximate the anatomical landmark. In some cases, the plurality ofintravascular imaging data sets (used for training the neural network)may include indications of key artery locations such as proximalreference, minimum lumen and distal reference and the one or moreintravascular images from the intravascular imaging event includes thesekey locations. In some cases, the predicted pressure values include apredicted pressure value proximate the key locations.

In some instances, outputting the one or more intravascular images incombination with the predicted hemodynamic values may include displayingthe one or more intravascular images and the predicted hemodynamicvalues on a graphical user interface of a signal processing unit.Displaying the one or more intravascular images and the predictedhemodynamic values on a graphical user interface of a signal processingunit may include displaying a fully co-registered display of thepredicted hemodynamic values with the intravascular images. In someinstances, displaying the one or more intravascular images and thepredicted hemodynamic values on a graphical user interface of a signalprocessing unit may include displaying a fully tri-registered display ofthe predicted hemodynamic values with the intravascular images and acorresponding extravascular image.

FIG. 3 is a flow diagram showing an illustrative method 38 forprocessing imaging data. The method 38 includes providing a plurality ofintravascular imaging data sets to a neural network (such as the neuralnetwork 12), wherein each intravascular imaging data set includesintravascular imaging data showing a portion of a blood vessel,co-registered to an extravascular image from a correspondingextravascular imaging data set, from a starting location to an endinglocation, as indicated at block 40. At least some of the plurality ofintravascular imaging data sets may include intravascular ultrasounddata. At least some of the plurality of intravascular imaging data setsmay include optical coherence tomography data. At least some of theplurality of extravascular imaging data sets may include fluoroscopicimage data. At least some of the plurality of extravascular imaging datasets may include angiographic image data.

A plurality of hemodynamic data sets are provided to the neural network,wherein each hemodynamic data set includes hemodynamic data from acorresponding portion of the blood vessel, co-registered to acorresponding extravascular image from the corresponding extravascularimaging data set, from a starting location to an ending location, asrepresented by one of the plurality of intravascular imaging data sets,as indicated at block 42. At least some of the plurality of hemodynamicdata sets may include pressure data obtained by any hyperemic ornon-hyperemic index. The neural network uses the provided intravascularimaging data sets and the corresponding provided hemodynamic data sets,from co-registration of each data set to the same extravascular image,to learn what hemodynamic data to expect for a given intravascularimaging data set, thereby training the neural network, as indicated atblock 44.

The method 38 includes performing in a new patient an intravascularimaging event in which an imaging element is translated within a bloodvessel from a starting location to an ending location in order toproduce one or more intravascular images, as indicated at block 46. Theneural network uses its training to predict hemodynamic valuescorresponding to the one or more intravascular images from theintravascular imaging event, as indicated at block 48. The one or moreintravascular images are outputted in combination with the predictedhemodynamic values, as indicated at block 50.

FIG. 4 is a flow diagram showing an illustrative method 52 of processingpatient imaging data. The method 52 includes obtaining intravascularimaging data from an intravascular imaging device including an imagingevent during a translation procedure during which the imaging element istranslated within a blood vessel from a starting location to an endinglocation, the intravascular imaging data including one or moreintravascular images, as indicated at block 54. The one or moreintravascular images are inputted into a trained neural network (such asthe trained neural network 24) in order to determine a predictedpressure reading for each of the one or more intravascular images, asindicated at block 56. A series of pressure values within the bloodvessel corresponding to an intravascular location of each of the one ormore extravascular images is calculated, as indicated at block 58. Apressure ratio is calculated based on the series of pressure values, asindicated at block 60.

In some cases, the method 52 may further include outputting theintravascular imaging data and the calculated pressure corresponding toa point within the blood vessel, as indicated at block 62. In someinstances, the method 52 may further include obtaining extravascularimaging data including one or more extravascular images, andco-registering the intravascular imaging data with the extravascularimaging data in order to determine an intravascular location of each ofthe one or more extravascular images, as indicated at block 64. Forexample, obtaining extravascular imaging data may include obtainingextravascular imaging data corresponding to the blood vessel from thestarting location to the ending location. In some instances, the method52 may further include also outputting the co-registered extravascularimaging data in combination with the intravascular imaging data and thecalculated pressure point corresponding to a point within the bloodvessel, as indicated at block 66.

In some instances, the intravascular imaging data includes intravascularultrasound data. In some cases, the intravascular imaging data includesoptical coherence tomography data. The extravascular imaging data mayinclude fluoroscopic image data, for example, or angiographic imagedata.

FIG. 5 is a flow diagram showing an illustrative method 68 of processingimaging data. The method 68 includes, from an IVUS pullback runproducing a plurality of IVUS frames, encoding physical features fromthe IVUS frames, as indicated at block 70. Physiology Resting Index(PRI) pullback data is collected, as indicated at block 72.Angiography-collected imaging data is collected, as indicated at block74. The angiography imaging is co-registered with the PRI pullback data,as indicated at block 76. The IVUS frames are co-registered with theangiography imaging data in order to co-register the PRI pullback datawith the IVUS frames, as indicated at block 78. The co-registered IVUSframes, angiography imaging data and PRI pullback data are used to traina neural network to predict PRI data based on a subsequent IVUS pullbackrun, as indicated at block 80. New IVUS pullback run data including aplurality of IVUS frames is subsequently provided to the neural networkin order to compute predicted PRI measurements for each IVUS frame, asindicated at block 82. In some cases, the method 68 may further includeco-registering the new IVUS pullback run data with a correspondingangiography run, as indicated at block 84.

FIGS. 6 through 8 provide schematic illustrations of illustrative modelsthat may be used in training the neural network 12. FIG. 6 is aschematic view of a model 86 that may be implemented within the neuralnetwork 12. The model 86 includes a number of inputs 86 a, including butnot limited to lumen borders, vessel borders, side-branches and bloodspeckle density. The inputs 86 a are provided via an N×1 86 b to aneural network block 86 c. The neural network 86 c outputs iFRpredictions 86 d. The model 86 employs a loss function 86 e. The model86 integrates a PRI hemodynamic model as an extra term in the lossfunction for increased PRI prediction unit accuracy. The model 86 takesinputs as a feature vector of derived lumen borders, vessel borders,side-branches, blood speckle density, cardiac cycle, etc. The inputfeature vector represents the variables of the hemodynamic PRI equation.

FIG. 7 is a schematic view of a model 88 that may be implemented withinthe neural network 12. IVUS (intravascular ultrasound) images 88 a areprovided to a CNN block 88 b. Embedded patches are provided to atransformer block 88 c. From there, signals pass to an MLP head 88 e andresult in IFR predictions 88 f. The model 88 processes IVUS imagesdirectly with CNNs and/or ViT to predict the PRI value.

FIG. 8 schematically shows a model 90 that is a combination of the model86 and the model 88. The outputs from the model 86 and the model 88 areprovided to an AVERAGING block 92. The output from the AVERAGING block92 is a final PRI 94. The model 90 provides a specific, accurate,efficient and real-time AI (artificial intelligence) model.

FIG. 9 is schematic depiction of an exemplary system 102 that may beused in conjunction with carrying out an embodiment of the presentdisclosure through obtaining and co-registering extravascular image data(e.g. angiogram/fluoroscopy) and intravascular image data (e.g. IVUS orOCT images). The system 102 may include an extravascular imagingsystem/sub-system 104 (e.g. angiography/fluoroscopy system) forobtaining/generating extravascular imaging data. The system 102 may alsoinclude an intravascular imaging system/sub-system 106 (e.g. IVUS orOCT) for obtaining/generating intravasular imaging data. The system 102may include a computer system/sub-system 130 including one or morecontroller or processor, memory and/or software configured to execute amethod for vascular imaging registration of the obtained extravascularimaging data and the obtained intravascular imaging data.

The extravascular imaging data may be radiological image data obtainedby the angiography/fluoroscopy system 104. Such angiography/fluoroscopysystems are generally well known in the art. The angiography/fluoroscopysystem 104 may include an angiographic table 110 that may be arranged toprovide sufficient space for the positioning of anangiography/fluoroscopy unit c-arm 114 in an operative position inrelation to a patient 100 on the table 110. Raw radiological image dataacquired by the angiography/fluoroscopy c-arm 114 may be passed to anextravascular data input port 118 via a transmission cable 116. Theinput port 118 may be a separate component or may be integrated into orbe part of the computer system/sub-system 130. Theangiography/fluoroscopy input port 118 may include a processor thatconverts the raw radiological image data received thereby intoextravascular image data (e.g angiographic/fluoroscopic image data), forexample, in the form of live video, DICOM, or a series of individualimages. The extravascular image data may be initially stored in memorywithin the input port 118, or may be stored within the computer 130. Ifthe input port 118 is a separate component from the computer 130, theextravascular image data may be transferred to the computer 130 throughthe cable 117 and into an input port in the computer 130. In somealternatives, the communications between the devices or processors maybe carried out via wireless communication, rather than by cables.

The intravascular imaging data may be, for example, IVUS data or OCTdata obtained by the intravascular imaging system/sub-system 106 (e.g.an IVUS or OCT system). Such IVUS and OCT systems are generally wellknown in the art. The intravascular sub-system 106 may include anintravascular imaging device such as an imaging catheter 120, forexample an IVUS or OCT catheter. The imaging device 120 is configured tobe inserted within the patient 100 so that its distal end, including adiagnostic assembly or probe 122 (e.g. an IVUS or OCT probe), is in thevicinity of a desired imaging location of a blood vessel. A radiopaquematerial or marker 123 located on or near the probe 122 may provideindicia of a current location of the probe 122 in a radiological image.

By way of example, in the case of IVUS intravascular imaging data, thediagnostic probe 122 generates ultrasound waves, and receives ultrasoundechoes representative of a region proximate the diagnostic probe 122.The probe 122 or catheter 120 may convert the ultrasound echoes intocorresponding signals, such as electrical or optical signals. Thecorresponding signals are transmitted along the length of the imagingcatheter 120 to a proximal connector 124. The proximal connector 124 ofthe catheter 120 is communicatively coupled to processing unit orcontrol module 126. IVUS versions of the probe 122 come in a variety ofconfigurations including single and multiple transducer elementarrangements. It should be understood that in the context of IVUS, atransducer may be considered an imaging element. In the case of multipletransducer element arrangements, an array of transducers is potentiallyarranged: linearly along a lengthwise axis of the imaging catheter 120,curvilinearly about the lengthwise axis of the catheter 120,circumferentially around the lengthwise axis, etc.

One example of an IVUS intravascular imaging catheter 120 is shown inFIGS. 10 and 11 . The imaging catheter 120 may include an elongate shaft170 having a proximal end region 172 and a distal end region 174. Theproximal hub or connector 124 may be coupled to or otherwise disposedadjacent to the proximal end region 172. A tip member 176 may be coupledto or otherwise disposed adjacent to the distal end region 174. The tipmember 176 may include a guidewire lumen, an atraumatic distal end, oneor more radiopaque markers, or other features. An imaging assembly 177may be disposed within the shaft 170. In general, the imaging assembly177 (which may include an imaging probe 122 including an imaging element182) may be used to capture/generate images of a blood vessel. In someinstances, the medical device may include devices or features similar tothose disclosed in U.S. Patent Application Pub. No. US 2012/0059241 andU.S. Patent Application Pub. No. US 2017/0164925, the entire disclosuresof which are herein incorporated by reference. In at least someinstances, the medical device 120 may resemble or include features thatresemble the OPTICROSS™ Imaging Catheter, commercially available fromBOSTON SCIENTIFIC, Marlborough, Mass.

As shown in FIG. 11 , the imaging assembly 177 may include a drive cableor shaft 178, an imaging probe 122 including a housing 180 and animaging element or transducer 182. The imaging probe 122 or housing 180may be coupled to the drive cable 178. The transducer 182 may berotatable or axially translatable relative to the shaft 170. Forexample, the drive cable 178 may be rotated or translated in order torotate or translate the transducer 182. The probe 122 or housing 180,for example, may include or be made of a radiopaque material or marker123, which may provide indicia of a current location of the probe 122 ina radiological image.

Referring back to FIG. 9 , by way of another example, the device 120 maybe an OCT catheter used to collect OCT intravascular data. The OCTcatheter 120 may include a diagnostic probe 122 that generates orpropagates a light beam that is directed at tissue, and a portion ofthis light that reflects from sub-surface features is collected and isrepresentative of a region proximate the diagnostic probe 122. In OCT,the diagnostic probe 122 will include an optical imager for delivery andcollection of the light. It should be understood that in the context ofOCT, the optical imager in the probe 122 may be considered an imagingelement. A technique called interferometry may be used to record theoptical path length of received photons allowing rejection of mostphotons that scatter multiple times before detection. Thus, OCT canbuild up images of thick samples by rejecting background signal whilecollecting light directly reflected from surfaces of interest. The probe122 or catheter 120 may transmit the optical or light signals along theshaft, or may convert light signals into corresponding signals, such aselectrical or optical signals, that may be transmitted along the lengthof the imaging catheter 120 to a proximal connector 124. The proximalconnector 124 of the catheter 120 is communicatively coupled to aprocessing unit or control module 126. The probe 122 or housing 180, mayinclude or be made of a radiopaque material or marker 123, which mayprovide indicia of a current location of the probe 122 in a radiologicalimage

Raw intravascular image data (e.g. raw IVUS or OCT data) may be acquiredby the imaging catheter 120 and may be passed to the control module 126,for example via connector 124. The control module 126 may be a separatecomponent or may be integrated into or be part of the computersystem/sub-system 130. The control module 126 may include a processorthat converts or is configured to convert the raw intravascular imagedata received via the catheter 120 into intravascular image data (e.gIVUS or OCT image data), for example, in the form of live video, DICOM,or a series of individual images. The intravascular imaging data mayinclude transverse cross-sectional images of vessel segments.Additionally, the intravascular imaging data may include longitudinalcross-sectional images corresponding to slices of a blood vessel takenalong the blood vessel's length. The control module 126 may beconsidered an input port for the computer system/subsystem 130, or maybe considered to be connected to an input port of the computer 130, forexample, via cable 119 or a wireless connection. The intravascular imagedata may be initially stored in memory within the control module 126, ormay be stored within memory in the computer system/subsystem 130. If thecontrol module 126 is a separate component from the computersystem/sub-system 130, the intravascular image data may be transferredto the computer 130, for example through the cable 119, and into aninput port in the computer 130. Alternatively, the communicationsbetween the devices or processors may be carried out via wirelesscommunication, rather than by cable 119.

The control module 126 may also include one or more components that maybe configured to operate the imaging device 120 or control thecollection of intravascular imaging data. For example, in the case of anIVUS system, the control module 126 may include one or more of aprocessor, a memory, a pulse generator, a motor drive unit, or adisplay. As another example, in the case of an OCT system, the controlmodule 126 may include one or more of a processor, a memory, a lightsource, an interferometer, optics, a motor drive unit, or a display. Insome cases, the control module 126 may be or include a motor drive unitthat is configured to control movement of the imaging catheter 120. Sucha motor drive unit may control rotation or translation of the imagingcatheter 120 or components thereof In some instances, the control module126 or motor drive unit may include an automatic translation system thatmay be configured to translate the imaging catheter 120 in acontrolled/measured matter within the patient 100. Such an automatictranslation system may be used such that during a translation procedure,the imaging catheter 120 (including an imaging element) is translatedwithin the blood vessel from a starting location to an ending locationat a constant or known speed. (e.g. the imaging catheter 120 istranslated at a specific rate for a known amount of time). In otherembodiments, the translation may be done manually. Translationprocedures may be, for example, a “pullback” procedure (where thecatheter 120 is pulled through the vessel) or a “push-through” procedure(where the catheter 120 is pushed through the vessel). The controlmodule 126 may also be configured from or include hardware and softwareconfigured to control intravascular imaging and data collection. Forexample, the control module 126 may include control features to turnon/off imaging or data collection from/to the catheter 120.

The computer system/sub-system 130 can include one or more controller orprocessor, one or more memory, one or more input port, one or moreoutput port and/or one or more user interface. The computer 130 obtainsor is configured to obtain intravascular image data from or through theintravascular imaging system/sub-system 106 (e.g. IVUS or OCT) andextravascular image data from or through the extravascular imagingsystem/sub-system 104 (e.g. angiography/fluoroscopy system). Thecomputer 130, or the components thereof, can include software andhardware designed to be integrated into standard catheterizationprocedures and automatically acquire both extravascular imaging data(e.g. angiography/fluoroscopy) and intravascular imaging data (e.g. IVUSor OCT) through image or video acquisition.

The computer system/sub-system 130, or the components thereof, caninclude software or hardware that is configured to execute a method forvascular imaging co-registration of the obtained extravascular imagingdata and the obtained intravascular imaging data. In that context, thecomputer 130 may include computer readable instructions or software toexecute the method for vascular imaging co-registration as disclosedherein. For example, in some respects the computer may include aprocessor or a memory which includes software including program codecausing the computer to execute the method for vascular imagingco-registration as disclosed herein. For example, the computer/computingdevice can include a processor or memory including instructionsexecutable by the processor to perform the method for vascular imagingco-registration as disclosed herein. In that context, it can also beappreciated that also disclosed herein is a computer readable mediumhaving stored thereon in a non-transitory state a program code for useby the computer/computing device 130, the program code causing thecomputing device 130 to execute the method for vascular imagingco-registration as disclosed herein. Additionally, thecomputer/computing device 130 may be part of or include a system forintravascular imaging registration that includes one or more input portfor receiving imaging data; one or more output port; and a controller incommunication with the input port and the output port, the controllerconfigured to execute the method for intravascular imaging registrationas disclosed herein.

The computer system/sub-system 130 can also include software andhardware that is configured for rendering or displaying imaging,including, for example, extravascular imaging or intravascular imagingderived from the received image data or co-registration method. In somecases, the computer 130 or software can be configured to render bothextravascular imaging and intravascular imaging on a single display. Inthat regard, the system may include a display 150 configured forsimultaneously displaying extravascular image data and intravascularimage data rendered by the computer 130. The display 150 may be part ofthe computer system 130 or may be a separate component in communicationwith the computer system 130, for example through an output port on thecomputer 130 and a transmission cable 121. In some other cases, however,the communication through the output port may be wireless, rather thanby cable. In some examples, the computer 130 or display 150 may beconfigured to simultaneously provide an angiogram, an IVUS transverseplane view, and an IVUS longitudinal plane view, which may or may notall be co-registered. In other examples, the display may be configuredto simultaneously provide an angiogram, an OCT transverse plane view,and an OCT longitudinal plane view, which may or may not beco-registered.

The computer system/sub-system 130 can also include one or moreadditional output ports for transferring data to other devices. Forexample, the computer can include an output port to transfer data to adata archive or memory 131. The computer system/sub-system 130 can alsoinclude a user interface that may include software and hardware that isconfigured for allowing an operator to use or interact with the system.

The components of the system 102 may be used cooperatively during avascular imaging method or procedure that involves the collection ofextravascular imaging data and intravascular imaging data during atranslation procedure. In the context of performing such a procedure,and obtaining the requisite imaging data, an example method forintravascular imaging registration may be executed or performed.

For example, the patient 100 may be arranged on the table 110 forextravascular imaging of a portion of a blood vessel of interest. Thepatient 100 or the table may be arranged or adjusted to provide for thedesired view of the vessel of interest, in preparation for thecollection of extravascular imaging data. Additionally, theintravascular imaging catheter 120 may be introduced intravascularlyinto the portion of the blood vessel of interest, in preparation for atranslation procedure to collect intravascular imaging data. Theintravascular imaging catheter 120 can be navigated, and positioned(often under fluoroscopy) within the vessel such that the imagingelement is located at a desired starting location for the translationprocedure. A guide catheter may be used to aid in navigation. Once inthe proper position, a translation procedure may be executed orperformed. Before or during the translation procedure, requisiteextravascular and intravascular imaging data may be obtained. In thiscontext, or as part of this process, an example method for vascularimaging co-registration or registration may be executed or performed.

Additional details regarding co-registering intravascular imaging datawith extravascular imaging data may be found in U.S. Ser. No.63/157,427, filed Mar. 5, 2021, which application is incorporated byreference herein in its entirety.

In some cases, the hemodynamic data may include pressure data. As anexample, FFR (fractional flow reserve) data may be obtained thatcompares pressure measured in the aorta with pressure measuredelsewhere, such as in the coronary arteries. If there are no blockagesor anything else impeding blood flow through the coronary arteries, thenthe pressure measured within the coronary arteries would be expected tobe the same as that measured in the aorta. The FFR can be considered asbeing a fraction of the two pressure values. If the fraction is belowone (1), this means that the pressure within the coronary arterycurrently being tested is lower than the aortic pressure. This canindicate a blockage or other impediment to blood flow within thatparticular coronary artery. In some cases, hemodynamic data such as FFR(fractional flow reserve) data may be co-registered with extravascularimaging data such as angiographic data. Intravascular imaging data suchas IVUS (intravascular ultrasound) may be co-registered with the sameextravascular imaging data in order to obtain both hemodynamic andintravascular imaging data for each location of interest within theextravascular imaging data.

It should be understood that this disclosure is, in many respects, onlyillustrative. Changes may be made in details, particularly in matters ofshape, size, and arrangement of steps without exceeding the scope of thedisclosure. This may include, to the extent that it is appropriate, theuse of any of the features of one example embodiment being used in otherembodiments. The invention's scope is, of course, defined in thelanguage in which the appended claims are expressed.

What is claimed is:
 1. A method for estimating patient hemodynamic data,the method comprising: training a neural network, where training theneural network comprises: providing a plurality of extravascular imagingdata sets to the neural network; providing a plurality of intravascularimaging data sets to the neural network, each intravascular imaging dataset including intravascular imaging data showing a portion of a bloodvessel from a starting location to an ending location, eachintravascular imaging data set co-registered to a correspondingextravascular imaging data set of the plurality of extravascular imagingdata sets; providing a plurality of hemodynamic data sets to the neuralnetwork, each hemodynamic data set co-registered with the correspondingextravascular imaging data set of the plurality of extravascular imagingdata sets; the neural network using the provided plurality ofintravascular imaging data sets and the provided plurality ofhemodynamic data sets, each co-registered with the correspondingextravascular imaging data set to learn what hemodynamic data to expectfor a given intravascular imaging data set, thereby creating a trainedneural network; using the trained neural network with a subsequentpatient, comprising: performing an intravascular imaging event in whichan intravascular imaging element is translated within a blood vessel ofthe patient from a starting location to an ending location in order toproduce one or more intravascular images; the trained neural networkusing its training to predict hemodynamic values corresponding to theone or more intravascular images from the intravascular imaging event;and outputting the one or more intravascular images in combination withthe predicted hemodynamic values.
 2. The method of claim 1, wherein atleast some of the plurality of intravascular imaging data sets providedwhile training the neural network comprise intravascular ultrasound dataor optical coherence tomography data.
 3. The method of claim 1, whereinat least some of the plurality of extravascular imaging data setsprovided while training the neural network comprise fluoroscopic imagedata or angiographic image data.
 4. The method of claim 1, wherein atleast some of the plurality of hemodynamic data sets provided whiletraining the neural network comprise pressure data obtained by anyhyperemic or non-hyperemic index.
 5. The method of claim 1, wherein atleast some of the plurality of intravascular imaging data sets and atleast some of the corresponding hemodynamic data sets are co-registeredusing their corresponding points in 2D or 3D space on the correspondingextravascular imaging data set.
 6. The method of claim 1, wherein theneural network comprises one or more of an ensemble of neural networks,a CNN (convoluted neural network) with transformers or a multi-layerneural network.
 7. The method of claim 1, wherein at least some of theplurality of intravascular imaging data sets provided while training theneural network include quantitative data such as lumen borders, vesselborders, side-branch borders, blood speckle density and cardiac cycleparameters; and the quantitative data is used in training the neuralnetwork.
 8. The method of claim 1, wherein: the one or moreintravascular images from the intravascular imaging event include ananatomical landmark; and the predicted hemodynamic values include apredicted pressure value proximate the anatomical landmark.
 9. Themethod of claim 1, wherein outputting the one or more intravascularimages in combination with the predicted hemodynamic values comprisesdisplaying the one or more intravascular images and the predictedhemodynamic values on a graphical user interface of a signal processingunit.
 10. The method of claim 9, wherein displaying the one or moreintravascular images and the predicted hemodynamic values on a graphicaluser interface of a signal processing unit comprises displaying a fullyco-registered display of the predicted hemodynamic values with theintravascular images.
 11. The method of claim 9, wherein displaying theone or more intravascular images and the predicted hemodynamic values ona graphical user interface of a signal processing unit comprisesdisplaying a fully tri-registered display of the predicted hemodynamicvalues with the intravascular images and a corresponding extravascularimage.
 12. A method for processing imaging data, the method comprising:providing a plurality of intravascular imaging data sets to a neuralnetwork, wherein each intravascular imaging data set includesintravascular imaging data showing a portion of a blood vessel,co-registered to an extravascular image from a correspondingextravascular imaging data set, from a starting location to an endinglocation; providing a plurality of hemodynamic data sets to the neuralnetwork, wherein each hemodynamic data set includes hemodynamic datafrom a corresponding portion of the blood vessel, co-registered to acorresponding extravascular image from the corresponding extravascularimaging data set, from a starting location to an ending location, asrepresented by one of the plurality of intravascular imaging data sets;the neural network using the provided intravascular imaging data setsand the corresponding provided hemodynamic data sets, fromco-registration of each data set to the same extravascular image, tolearn what hemodynamic data to expect for a given intravascular imagingdata set, thereby training the neural network; performing in a newpatient an intravascular imaging event in which an imaging element istranslated within a blood vessel from a starting location to an endinglocation in order to produce one or more intravascular images; theneural network using its training to predict hemodynamic valuescorresponding to the one or more intravascular images from theintravascular imaging event; and outputting the one or moreintravascular images in combination with the predicted hemodynamicvalues.
 13. The method of claim 12, wherein at least some of theplurality of intravascular imaging data sets comprise intravascularultrasound data or optical coherence tomography data.
 14. The method ofclaim 12, wherein at least some of the plurality of extravascularimaging data sets comprise fluoroscopic image data or angiographic imagedata.
 15. The method of claim 12, wherein at least some of the pluralityof hemodynamic data sets comprise pressure data obtained by anyhyperemic or non-hyperemic index.
 16. A method for processing patientimaging data, the method comprising: obtaining intravascular imagingdata from an intravascular imaging device including an imaging eventduring a translation procedure during which the imaging element istranslated within a blood vessel from a starting location to an endinglocation, the intravascular imaging data including one or moreintravascular images; inputting the one or more intravascular imagesinto a trained neural network in order to determine a predicted pressurereading for each of the one or more intravascular images; calculating aseries of pressure values within the blood vessel corresponding to anintravascular location of each of the one or more extravascular images;and calculating a pressure ratio based on the series of pressure values.17. The method of claim 16, further comprising outputting: theintravascular imaging data; and the calculated pressure corresponding toa point within the blood vessel.
 18. The method of claim 16, furthercomprising: obtaining extravascular imaging data including one or moreextravascular images; co-registering the intravascular imaging data withthe extravascular imaging data in order to determine an intravascularlocation of each of the one or more extravascular images.
 19. The methodof claim 18, further comprising also outputting the co-registeredextravascular imaging data in combination with the intravascular imagingdata and the calculated pressure point corresponding to a point withinthe blood vessel.
 20. The method of claim 16, wherein obtainingextravascular imaging data comprises obtaining extravascular imagingdata corresponding to the blood vessel from the starting location to theending location.